Papers with selective mechanism
Selective Prefix Tuning for Pre-trained Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained models are time-consuming and memory-inefficient. |
| Approach: | They propose a method that inserts learnable vectors into each Transformer layer . they propose SL to encourage diversity in prefix tokens . |
| Outcome: | Extensive experiments validate the effectiveness of Prefix Tuning in sentence and token classification tasks. |
How Does Selective Mechanism Improve Self-Attention Networks? (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show that selective SANs outperform the standard SAN by paying more attention to content words that contribute to the meaning of the sentence. |
| Approach: | They propose to implement selective SANs with a flexible Gumbel-Softmax to improve word order encoding and structure modeling. |
| Outcome: | The proposed system outperforms the standard SANs on several representative NLP tasks including natural language inference, semantic role labelling, and machine translation. |
GraphCheck: Multipath Fact-Checking with Entity-Relationship Graphs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | GraphCheck is a framework for fact-checking complex claims that require multi-hop reasoning . Graphcheck excels in complex scenarios, but may be unnecessarily elaborate for simpler claims . |
| Approach: | They propose a framework that transforms claims into entity-relationship graphs for fact-checking . DP-GraphCheck employs a lightweight strategy selector to choose between direct prompting and GraphCheck adaptively. |
| Outcome: | The proposed framework outperforms existing methods in verification accuracy while achieving strong computational efficiency. |